A walk in the statistical mechanical formulation of neural networks

摘要

Neural networks are nowadays both powerful operational tools (e.g., forpattern recognition, data mining, error correction codes) and complextheoretical models on the focus of scientific investigation. As for theresearch branch, neural networks are handled and studied by psychologists,neurobiologists, engineers, mathematicians and theoretical physicists. Inparticular, in theoretical physics, the key instrument for the quantitativeanalysis of neural networks is statistical mechanics. From this perspective,here, we first review attractor networks: starting from ferromagnets andspin-glass models, we discuss the underlying philosophy and we recover thestrand paved by Hopfield, Amit-Gutfreund-Sompolinky. One step forward, wehighlight the structural equivalence between Hopfield networks (modelingretrieval) and Boltzmann machines (modeling learning), hence realizing a deepbridge linking two inseparable aspects of biological and robotic spontaneouscognition. As a sideline, in this walk we derive two alternative (with respectto the original Hebb proposal) ways to recover the Hebbian paradigm, stemmingfrom ferromagnets and from spin-glasses, respectively. Further, as these notesare thought of for an Engineering audience, we highlight also the mappingsbetween ferromagnets and operational amplifiers and between antiferromagnetsand flip-flops (as neural networks -built by op-amp and flip-flops- areparticular spin-glasses and the latter are indeed combinations of ferromagnetsand antiferromagnets), hoping that such a bridge plays as a concreteprescription to capture the beauty of robotics from the statistical mechanicalperspective.